<html>
<head><meta charset="utf-8"><title>async iteration semantics · t-lang · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/213817-t-lang/index.html">t-lang</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/213817-t-lang/topic/async.20iteration.20semantics.html">async iteration semantics</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="210496173"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/213817-t-lang/topic/async%20iteration%20semantics/near/210496173" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Yoshua Wuyts <a href="https://rust-lang.github.io/zulip_archive/stream/213817-t-lang/topic/async.20iteration.20semantics.html#210496173">(Sep 18 2020 at 09:25)</a>:</h4>
<p>Hi hi, I wrote a blog post to attempt to answer the question:  "Should async iteration syntax default to parallel or sequential semantics"? -- this came up again during the streams lang meeting a few weeks ago, and figured it was worth exploring.</p>
<p><a href="https://blog.yoshuawuyts.com/async-iteration/">https://blog.yoshuawuyts.com/async-iteration/</a></p>



<a name="210588180"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/213817-t-lang/topic/async%20iteration%20semantics/near/210588180" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> scottmcm <a href="https://rust-lang.github.io/zulip_archive/stream/213817-t-lang/topic/async.20iteration.20semantics.html#210588180">(Sep 19 2020 at 00:23)</a>:</h4>
<p>Hopefully at some point we'll get streaming iterators, too, which (in general) cannot be processed in parallel even if you wanted to.</p>



<a name="210589492"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/213817-t-lang/topic/async%20iteration%20semantics/near/210589492" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> oliver <a href="https://rust-lang.github.io/zulip_archive/stream/213817-t-lang/topic/async.20iteration.20semantics.html#210589492">(Sep 19 2020 at 00:52)</a>:</h4>
<p>tl;dr in general now, or in general ever?</p>



<a name="210607128"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/213817-t-lang/topic/async%20iteration%20semantics/near/210607128" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/213817-t-lang/topic/async.20iteration.20semantics.html#210607128">(Sep 19 2020 at 09:48)</a>:</h4>
<p>Streaming iterators invalidate an item when you fetch the next item. For instance, it may be borrowed from the stream itself.</p>



<a name="210607133"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/213817-t-lang/topic/async%20iteration%20semantics/near/210607133" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/213817-t-lang/topic/async.20iteration.20semantics.html#210607133">(Sep 19 2020 at 09:48)</a>:</h4>
<p>So with a streaming iterator, you couldn't process in parallel.</p>



<a name="210607140"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/213817-t-lang/topic/async%20iteration%20semantics/near/210607140" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/213817-t-lang/topic/async.20iteration.20semantics.html#210607140">(Sep 19 2020 at 09:49)</a>:</h4>
<p>(unless you make copies)</p>



<a name="210612136"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/213817-t-lang/topic/async%20iteration%20semantics/near/210612136" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Charles Lew <a href="https://rust-lang.github.io/zulip_archive/stream/213817-t-lang/topic/async.20iteration.20semantics.html#210612136">(Sep 19 2020 at 12:12)</a>:</h4>
<p>I think it actually depends on the <code>Item&lt;'iter&gt;</code> type, or how you use the <code>'iter</code> lifetime.   If it's <code>type Item&lt;'iter&gt; = &amp;'iter i32;</code> that means it invalidates when you fetch the next item, if it's <code>type Item&lt;'iter&gt; = i32;</code> That would be the same with a normal iterator.</p>



<a name="210634536"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/213817-t-lang/topic/async%20iteration%20semantics/near/210634536" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> scottmcm <a href="https://rust-lang.github.io/zulip_archive/stream/213817-t-lang/topic/async.20iteration.20semantics.html#210634536">(Sep 19 2020 at 20:46)</a>:</h4>
<p>Certainly it depends what exactly the type is.  But I wouldn't want <code>for</code> to fail to compile just because it's <code>type Item&lt;'iter&gt; = &amp;'iter mut State;</code>.</p>



<a name="210638905"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/213817-t-lang/topic/async%20iteration%20semantics/near/210638905" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Yoshua Wuyts <a href="https://rust-lang.github.io/zulip_archive/stream/213817-t-lang/topic/async.20iteration.20semantics.html#210638905">(Sep 19 2020 at 22:45)</a>:</h4>
<p>Oh that's a great point about streaming iterators!</p>



<a name="210640422"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/213817-t-lang/topic/async%20iteration%20semantics/near/210640422" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Yoshua Wuyts <a href="https://rust-lang.github.io/zulip_archive/stream/213817-t-lang/topic/async.20iteration.20semantics.html#210640422">(Sep 19 2020 at 23:31)</a>:</h4>
<p>Updated the post with a note on lending streams: <a href="https://blog.yoshuawuyts.com/async-iteration/#addendum-lending-streams">https://blog.yoshuawuyts.com/async-iteration/#addendum-lending-streams</a></p>



<a name="210640478"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/213817-t-lang/topic/async%20iteration%20semantics/near/210640478" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Yoshua Wuyts <a href="https://rust-lang.github.io/zulip_archive/stream/213817-t-lang/topic/async.20iteration.20semantics.html#210640478">(Sep 19 2020 at 23:33)</a>:</h4>
<p>(side note: the streams RFC proposed to name "streaming streams" -&gt; "lending streams")</p>



<a name="210641008"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/213817-t-lang/topic/async%20iteration%20semantics/near/210641008" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Matthias247 <a href="https://rust-lang.github.io/zulip_archive/stream/213817-t-lang/topic/async.20iteration.20semantics.html#210641008">(Sep 19 2020 at 23:52)</a>:</h4>
<blockquote>
<p>A large part of the argument in favor of parallel semantics by default for async iteration is that it is the most commonly desired semantics. </p>
</blockquote>
<p>To be honest - I haven't found a single example of where trivial "parallel iteration" would be desirable. And this while maintaining more than enough (async) and highly concurrent codebases.<br>
The blog post also really doesn't provide one. It agrees that for things like stream manipulation (e.g. zipping) it wouldn't be helpful. And for things like a TCP listener that spawns subtasks I generally need way more functionality (e.g. control of maximum concurrency, integration into a shutdown system, etc). The latter is ususally provided by a more sophisticated framework (that uses some version of <code>spawn</code> internally).  <code>FuturesUnordered</code> is a primitive version of that - production apps might have something different dependening on their use-case.</p>



<a name="210641057"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/213817-t-lang/topic/async%20iteration%20semantics/near/210641057" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Matthias247 <a href="https://rust-lang.github.io/zulip_archive/stream/213817-t-lang/topic/async.20iteration.20semantics.html#210641057">(Sep 19 2020 at 23:54)</a>:</h4>
<p>But actually I'm not even convinced that we need any special "async iteration" semantics. <code>while let something = something().await</code> works perfectly fine.</p>



<a name="213840088"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/213817-t-lang/topic/async%20iteration%20semantics/near/213840088" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/213817-t-lang/topic/async.20iteration.20semantics.html#213840088">(Oct 19 2020 at 20:22)</a>:</h4>
<p><span class="user-mention" data-user-id="204219">@Matthias247</span> </p>
<blockquote>
<p>To be honest - I haven't found a single example of where trivial "parallel iteration" would be desirable. And this while maintaining more than enough (async) and highly concurrent codebases.</p>
</blockquote>
<p>just to make sure I understand you, are you saying that you typically want <em>sequential</em> semantics?</p>



<a name="214011311"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/213817-t-lang/topic/async%20iteration%20semantics/near/214011311" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Matthias247 <a href="https://rust-lang.github.io/zulip_archive/stream/213817-t-lang/topic/async.20iteration.20semantics.html#214011311">(Oct 21 2020 at 06:10)</a>:</h4>
<blockquote>
<blockquote>
<p>nikomatsakis: @Matthias247</p>
</blockquote>
<p>just to make sure I understand you, are you saying that you typically want sequential semantics?</p>
</blockquote>
<p>That's right. I only had use-cases for sequential semantics so far. There are some use-cases where I would <code>spawn</code> afterwards. But these are rare (&lt; 5%), and typically need some extra work to make sure parallelism is limited and there is an overall cancellation + wait mechanism.</p>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>